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MEMORANDUM  FOR  DIRECTOR,  DEFENSE  RESEARCH  AND  ENGINEERING 


SUBJECT:  Audit  Report  on  the  Joint  Cross-Service  GrouPj^L^at°rf s  1995  BaSe 
Realignment  and  Closure  Process  (Project  No.  4CG-5016.07) 


Introduction 


We  are  providing  this  final  report  for  your  information  and  use.  This  report  is 
one  in  a  series  of  reports  that  discusses  the  review  of  the  Joint  Cross-Service 
Groups'  (JCSGs)  implementation  of  the  internal  control  plan  developed  by  die 
1995 ^Defense  Base  Realignment  and  Closure  (BRAC)  Steering  Group  (the 
Steering  Group)  for  managing  the  identification  of  DoD  cross-Service 
opportunities.  The  Deputy  Secretary  of  Defense  directed  the  Inspector  General, 
DoD,  to  review  the  adequacy  and  implementation  of  the  internal  control  phno 
this  process.  The  report  discusses  the  adequacy  of  the  Joint  Cross-Service 
Group  for  Laboratories  implementation  of  the  internal  control  plan. 


Audit  Results 


The  JCSG  for  Laboratories  implementation  of  the  internal  control  plan  was 
generally  effective.  We  did  not  identify  any  significant  deficiencies,  errors,  or 
inconsistencies  in  data  consolidation. 


Audit  Objectives 


The  overall  audit  objective  was  to  assess  the  adequacy  of  the  Steering  Group 
internal  control  plan.  The  specific  objective  of  this  audit  was  to  determine 
whether  the  JCSG  for  Laboratories  adequately  implemented  the  internal  control 
plan.  A  summary  report  will  discuss  the  overall  audit  objective. 


Scope  and  Methodology 

We  reviewed  the  JCSG  for  Laboratories  process  for  collecting  and  analyzing 
1995  BRAC  data  for  laboratories.  We  did  not  review  the  data  collection 
process  of  the  Military  Departments. 


We  attended  JCSG  for  Laboratories  meetings  and  reviewed  the  formal  minutes 
and  briefing  charts  of  the  meetings. 


We  verified  the  accuracy  of  the  JCSG  for  Laboratories  consolidation  of  data 
submitted  by  the  Military  Departments. 


K S 

pSessL  dL,  See  tnctare  i  for  a  complete  list  of  the  organizations  we 
visited  or  contacted. 

Internal  Control  Plan 

°,”  r,«  .^"ve^me 

closures  and  realignments  would  be  based. 

The  internal  control  plan  established  two  principal  mechanisms  to  control  the 
process:  organization  and  documentation. 

Implementation  of  Organization*  C»ntrob.  ^O^amzatfonal  ^ntt* 

controls. 

implementation  of 

feZ,  e„»g“ S'  S' -  Ss,  technical  experts, 
and  access  to  records. 

^vembml  lK 

memorandum,  "1995  Base 

Realignment  and  Closures,"  January  7,  1994. 


Prior  Audits  and  Other  Reviews 

No  previous  audit  coverage  of  the  JCSG  for  Laboratories  has  occurred. 

Audit  Background 

T  n  1 QQ/L  TVniitv  Secretary  of  Defense  memorandum  established 

$crp™irs  for  seiec,ing  bases 
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<*  internal 

control  plan. 

Review  Group  Authorities.  The  Review  Group  oversaw  the  entire  1995 
wrap  nrocess”  The  Review  Group  was  chaired  by  the  Deputy  Secretary  ot 

^sskssssssss 

o  reviewing  1995  BRAC  analysis  policies  and  procedures, 
o  reviewing  1995  BRAC  excess  capacity  analyses, 

n  establishing  1995  BRAC  closure  or  realignment  alternatives  Mid 
numerical  e“efpadty  reduction  targets  for  considerat.cn  by  the  DoD 
Components,  and 

o  making  recommendations  to  the  Secretary  of  Defense. 

SSM&SaBSSSf^g 

D^feme'fEcOTomic  Security)  and  tL  composed  «*: the 
Military  Departments,  the  JCSGs,  and  various  other  DoD  Components. 

iper  Authorities  The  JCSGs  were  established  in  six  areas  with  significant 
potential ^or^rosT-Sen'ice  impact  in  the  1995  BRAC  process.  The  six  JCSGs 

areas  were: 


o  Depot  Maintenance; 
o  Test  and  Evaluation; 


o  Laboratories; 

o  Medical  Treatment  Facilities,  including  Graduate  Medical  Education, 
o  Undergraduate  Pilot  Training;  and 


o  Economic  Impact. 

Thf>  JCSGs  were  chaired  by  senior  DoD  officials,  with  members  from  each  of 
DepamS  L  other  DoD  offices,  as  considered  appropriate  by 

the  chairperson. 
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jCSG  Responsibilities.  The  JCSGs  (excluding  the  JCSG  for  Economic 
Impact)  were  tasked  to  perform  the  following  functions. 

n  Fstablish  guidelines,  standards,  assumptions,  measures  of  merit  data 
elemen^S  iSteS  Shedu.es  for  cross-Service  anaiysis  of  the  conunon- 

support  area. 


o  Perform  an  excess-capacity  analysis. 


J*.  V1AVA***  -  M.  ~ 

o  Develop  closure  or  realignment  alternatives  and  numerical  excess- 
caDacity  reduction  targets  for  Military  Department  consideration. 


o  Analyze  cross-Service  tradeoffs. 

JCSG  Process.  To  fulfill  its  responsibilities,  each  group  generally 
performed  the  following  tasks. 

o  Develop  a  methodology  for  analyzing  cross-Service  opportunities. 

o  Develop  data  call  requirements  to  use  in  the  analysis. 

o  Determine  capacity  requirements  and  functional  value  and  prepare 
optimization  scenarios. 

o  Analyze  the  results  for  operational  feasibility. 

o  Recommend  cross-Service  opportunities  to  the  Military  Departments. 


Discussion 


The  JCSG  for  Laboratories  was  chaired  by  the  Director,  Defense  Research  and 

We  reviewed  the  JCSG  for  Laboratories  implementation  of  the  plan. 

Data  Call  Information  and  Collection.  The  JCSG  for  Laboratories  developed 
a  data  rail  that  tasked  the  Military  Departments  to  collect  capacity  data  for  eac 

ttSSSs-- 

p“s  g— fjsr  Teco.w 

miSoMocatiou,  personal,  facilities  and  equipment,  and  expans, on 

potential. 


The  JCSG  for  Laboratories  designated  a  laboratory  study  team,  composed  of  the 
alternate  members  of  the  JCSG,  to  develop  the  measures  of  merit.  The 
measures  of  merit  for  laboratories  were  reduction  of  laboratory  infrastructure, 
return  on  investment,  and  military  value. 

The  JCSG  for  Laboratories  developed  an  analysis  plan  that  defined  how  the  data 
would  be  used  to  determine: 

o  functional  capacity  (the  maximum  workload  capacity  of  an  activity), 

o  functional  requirement  (the  projected  workload  requirement  for  a 
given  common  support  function),  and 

o  functional  value  (a  measure  of  the  value  of  a  common  support 
function  performed  at  an  activity). 

The  analysis  plan  also  addressed  how  other  analytical  tools  would  be  used  to 
develop  alternatives. 

Functional  value  is  a  measure  of  the  capability  to  perform  work  and  the  quality 
of  work  performed  in  a  specific  function  at  a  specific  activity.  The  JCSG  for 
Laboratories  computed  functional  value  by  evaluating  and  scoring  the  certified 
cross-Service  data  against  the  functional  value  measures  and  then  applying  the 
functional  value  weights  in  the  functional  value  model. 

The  functional  value  rating  system  was  developed  by  reviewing  the  data  call 
elements  for  applicability,  comparability,  and  ability  to  act  as  a  value 
discriminator  in  calculating  functional  value.  The  JCSG  for  Laboratories 
developed  measures  to  limit  the  amount  of  subjective  evaluation  required. 
Some  of  the  data  submitted  by  the  Military  Departments  were  used  in  the 
functional  value  calculation,  and  other  data  were  used  as  background  or  for  "fit 
check"  evaluation  of  the  alternatives  proposed.  (Fit  check  is  a  part  of  the 
overall  process  for  analyzing  operational  feasibility  of  alternatives  derived  from 
the  optimization  model). 

Functional  value  weights  were  developed  by  the  JCSG  for  Laboratories  based 
on  recommendations  from  the  Laboratory  JCSG  Working  Group.  Relative 
importance  was  derived  by  comparing  measures  with  each  other  and  across 
target  categories,  from  which  several  of  the  measures  could  be  collected. 

The  functional  value  model  used  Decision-PAD,  a  commercially  available 
product.  Decision-PAD  was  programmed  to  yield,  for  each  measure,  an 
adjusted  score  between  1  and  100  for  each  common  support  function  and 
activity  combination.  Each  adjusted  score  was  multiplied  by  its  associated 
weight,  and  then  all  were  summed  to  yield  a  single  functional  value  (between  1 
and  100)  for  each  common  support  function  and  activity. 

The  chairman  of  the  JCSG  for  Laboratories  transmitted  the  guidance  package  to 
the  Military  Departments  on  March  30,  1994.  This  guidance  package 
established  the  guidelines,  standards,  assumptions,  measures  of  merit,  data 
elements,  and  milestone  schedule  for  the  DoD  Components  to  conduct  cross- 
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Service  analyses  of  common  support  functions  performed  by  laboratories.  The 
guidance  package  also  provided  a  questionnaire  to  ensure  common,  comparable 
data  was  gathered.  This  document  required  that  the  data  meet  the  certification 
requirements  of  1995  BRAC.  Military  Departments  were  required  to  submit  the 
responses  to  the  data  call  by  July  1,  1994. 

The  JCSG  for  Laboratories,  consistent  with  the  internal  control  plan, 
documented  in  the  analysis  plan  a  disciplined  and  controlled  process  for  scoring 
and  evaluating  the  data  to  preserve  the  integrity  of  the  process  and  control 
access  to  the  data. 

The  Inspector  General,  DoD,  review  consisted  of  monitoring  the  development 
process,  determining  whether  the  decision  process  was  adequately  documented, 
and  determining  whether  the  data  requirements  assured  consistency  of  the  data 
collected.  Auditor  personnel  attended  each  formal  meeting  of  the  JCSG  for 
Laboratories  and  were  resident  at  the  Defense  Test  and  Evaluation  Center, 
located  at  the  Institute  for  Defense  Analyses,  Alexandria,  Virginia,  to  monitor 
the  activities  and  ensure  compliance  with  the  approved  internal  control  and 
management  control  plans.  We  participated  in  the  inventory  of  the  Military 
Departments  data  call  responses. 

The  JCSG  for  Laboratories  documented  its  decisions  on  the  criteria  to  be  used 
in  the  data  requirements  in  minutes  and  in  various  briefing  packages. 

Certification  of  the  Data  Call  Results.  The  January  7,  1994,  Deputy 
Secretary  of  Defense  memorandum  required  DoD  Components  to  certify  the 
data  collected  by  the  Military  Departments  were  accurate,  complete,  and 
consistent.  The  JCSG  for  Laboratories  prepared  a  Management  Control  Plan 
for  the  Laboratories  Scoring  Team  to  document  procedures  for  data 
configuration  management  and  control.  This  document  designated  a  data 
configuration  control  manager  to  be  the  recipient  of  all  data  for  the  JCSG  for 
Laboratories.  The  Steering  Group  requested  the  Inspector  General,  DoD, 
auditors,  as  independent  observers,  to  assist  in  the  data  administration.  The 
procedures  ensured  complete  traceability  existed  for  all  data  received.  We 
maintained  the  original  data  responses  as  the  control  copy.  During  the 
inventory  of  each  Military  Department's  data  call  responses,  we  examined  each 
response  for  the  proper  certifications.  The  JCSG  analysis  plan  included  a 
documented  process  to  ensure  the  integrity  of  the  process  and  to  control  access 
to  the  data. 

We  identified  a  potential  deficiency  in  the  certification  process  at  the  Armed 
Forces  Radiobiology  Research  Institute,  Bethesda,  Maryland.  The  institute  is  a 
subordinate  command  of  the  Uniformed  Services  University  of  Health  Sciences 
reporting  to  the  Assistant  Secretary  of  Defense  (Health  Affairs).  The  other 
DoD  laboratories  responding  to  the  data  call  are  components  of  the  Military 
Departments;  their  data  submissions  had  Military  Department  audit  agency 
oversight.  Because  the  Armed  Forces  Radiobiology  Research  Institute  is  not 
affiliated  with  a  specific  Military  Department,  we  reviewed  the  data  collection 
process  in  response  to  the  data  call  to  comply  with  the  oversight  requirement. 
We  issued  Inspector  General,  DoD,  Report  No.  95-020,  "Armed  Forces 
Radiobiology  Research  Institute  1995  Defense  Base  Realignment  and  Closure 
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Data  Call  Response,"  on  October  31,  1994.  The  report  states  that  the  Armed 
Forces  Radiobiology  Research  Institute  maintained  sufficient  supporting 
documentation  for  all  elements  reviewed.  The  data  provided  to  the  JCSG  for 
Laboratories  were  adequately  supported  and  consistent  with  the  JCSG  guidance. 

Record  Keeping.  The  JCSG  for  Laboratories  documented  the  consolidation  of 
data  processes  in  the  Management  Control  Plan  for  Laboratories  Scoring  Team 
Space  and  in  the  analysis  plan.  Evaluation  criteria  scoring  sheets  were  used  to 
record  the  scoring.  The  data  were  scored  by  teams  made  up  of  one  member 
from  each  Military  Department.  Procedures  called  for  at  least  two  reviews  of 
the  data  to  serve  as  an  indication  of  the  consistency  with  which  activities 
interpreted  the  data  call  questions.  The  scoring  sheets  were  used  to  convert  the 
certified  data  provided  by  the  Military  Departments  into  a  form  that  would  later 
be  entered  into  the  Decision-Pads  model. 

We  verified  the  accuracy  of  the  data  in  the  spreadsheets  for  calculating 
functional  capacity  and  functional  requirement  by  verifying  that  the  data  from 
the  score  sheets  were  transferred  accurately  and  completely  to  the  spreadsheets. 
We  performed  a  100-percent  analysis  of  the  data  entry  to  ensure  that  the  latest 
data  were  used  for  the  programs  and  that,  as  revised  information  was  received, 
the  computer  program  was  updated.  We  also  ensured  that  all  of  the  data  call 
responses  were  used  in  the  analysis. 

Oral  Briefings.  Oral  briefings  were  presented  to  the  Chairman  at  29  meetings 
of  the  JCSG  for  Laboratories.  During  the  meetings,  status,  problems,  and 
recommendations  were  presented  to  the  group.  We  attended  the  meetings  and 
reviewed  the  minutes  to  determine  completeness  and  accuracy.  The  minutes 
and  attached  briefing  charts  were  maintained  in  the  Office  of  Director,  Base 
Closure  and  Utilization,  and  were  found  to  be  an  accurate  representation  of  the 
meetings. 

Outside  Studies  and  Technical  Experts.  The  JCSG  for  Laboratories  did  not 
use  outside  studies  or  technical  experts  in  its  process  to  implement 
documentation  controls. 

Access  to  Records.  The  JCSG  for  Laboratories  maintained  security  over  the 
data  packages  received  from  the  Military  Departments  to  ensure  the  integrity  of 
the  data.  A  dedicated,  secure,  work  area  was  arranged  at  the  Defense  Test  and 
Evaluation  Center,  at  the  Institute  for  Defense  Analyses,  for  JCSG  for 
Laboratories  personnel.  The  suite  of  offices  had  a  cipher  lock  on  the  door,  and 
classified  material  safes  with  combination  locks  were  used  to  store  the  data. 
Data  processing  equipment  with  removable  hard  drives  were  also  used. 

Security  procedures  were  documented  in  the  Management  Control  Plan  for 
Laboratories  Scoring  Team  Space.  The  assigned  data  administrator  was 
responsible  for  establishing  and  maintaining  an  access  list  to  the  facility.  He 
also  issued  and  controlled  the  combinations  for  the  facility  cipher  lock.  With 
our  assistance,  he  controlled  access  to  the  data  call  responses  and  to  the 
computer  hard  drives. 
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Recommendations  to  the  Military  Departments 

The  JCSG  for  Laboratories  transmitted  its  alternatives  for  Military  Department 
consideration  in  four  separate  documents.  The  alternatives  were  sent  on 
November  1,  4,  21,  and  29,  1994.  The  results  were  organized  by  common 
support  function  and  described  the  activities  suggested  for  continued 
participation  in  a  specific  common  support-function  life  cycle.  Also  enclosed  in 
the  package  was  a  guide  describing  the  information  contained  in  the  enclosed  set 
of  alternatives.  The  recommendations  package  included  copies  of  the  model 
outputs  of  each  consideration. 


Management  Comments 

We  provided  a  draft  of  this  report  to  the  Deputy  Assistant  Secretary  of  Defense 
(Installations)  on  March  15,  1995.  Because  the  report  contained  no  findings  or 
recommendations,  written  comments  were  not  required.  The  Deputy  Assistant 
Secretary  of  Defense  (Installations)  concurred  with  the  report.  Full  text  of 
Management  Comments  is  in  Enclosure  3. 

We  appreciate  the  courtesies  extended  to  the  audit  staff.  If  you  have  any 
questions  on  this  report,  please  contact  Mr.  Raymond  A.  Spencer,  Audit 
Program  Director,  at  (703)  604-9070  (DSN  664-9070)  or  Ms.  Nancee  K. 
LaBute,  Acting  Audit  Project  Manager,  at  (703)  604-9054  (DSN  664-9054). 
Enclosure  2  lists  the  distribution  of  this  report. 


o8~6iM>tL  % 


David  K.  Steensma 
Deputy  Assistant  Inspector  General 
for  Auditing 


Enclosures 
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Organizations  Visited  or  Contacted 


Office  of  the  Secretary  of  Defense 

Assistant  Secretary  of  Defense  (Economic  Security),  Washington,  DC 
Director,  Base  Closure  and  Utilization,  Washington,  DC 
Director,  Defense  Research  and  Engineering,  Washington,  DC 
Armed  Forces  Radiobiology  Research  Institute,  Bethesda,  MD 


Report  Distribution 


Office  of  the  Secretary  of  Defense 

Under  Secretary  of  Defense  (Comptroller) 

Assistant  Secretary  of  Defense  (Economic  Security) 
Deputy  Assistant  Secretary  of  Defense  (Installations) 
Director,  Base  Closure  and  Utilization 
Director,  Defense  Research  and  Engineering 
Assistant  to  the  Secretary  of  Defense  (Public  Affairs) 


Department  of  the  Army 

Secretary  of  the  Army 

Auditor  General,  Department  of  the  Army 


Department  of  the  Navy 

Secretary  of  the  Navy 

Assistant  Secretary  of  the  Navy  (Financial  Management) 
Auditor  General,  Department  of  the  Navy 


Department  of  the  Air  Force 

Secretary  of  the  Air  Force 

Assistant  Secretary  of  the  Air  Force  (Financial  Management  and  Comptroller) 
Auditor  General,  Department  of  the  Air  Force 


Other  Defense  Organizations 

Director,  Defense  Contract  Audit  Agency 
Director,  Defense  Logistics  Agency 

Director,  Defense  Logistics  Studies  Information  Exchange 
Director,  National  Security  Agency 

Inspector  General,  National  Security  Agency 
Inspector  General,  Central  Imagery  Office 
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Enclosure  2 
(Page  1  of  2) 


Non-Defense  Federal  Organizations  and  Individuals 

Office  of  Management  and  Budget 

Technical  Information  Center,  National  Security  and  International  Affairs  Division, 
General  Accounting  Office 

Chairman  and  ranking  minority  member  of  each  of  the  following  congressional 
committees  and  subcommittees: 

Senate  Committee  on  Appropriations 

Senate  Subcommittee  on  Defense,  Committee  on  Appropriations 
Senate  Committee  on  Armed  Services 
Senate  Committee  on  Governmental  Affairs 
House  Committee  on  Appropriations 

House  Subcommittee  on  National  Security,  Committee  on  Appropriations 
House  Committee  on  Government  Reform  and  Oversight 
House  Subcommittee  on  National  Security,  International  Affairs,  and  Criminal 
Justice,  Committee  on  Government  Reform  and  Oversight 
House  Committee  on  National  Security 


W 


Enclosure  2 
(Page  2  of  2) 


Office  of  the  Assistant  Secretary  of  Defense 
Comments 


ECONOMIC  ICCUKITV 


OFFICE  OF  THE  ASSISTANT  SECRETARY  OF  DEFENSE 
3300  DEFENSE  PENTAGON 
WASHINGTON.  DC  20301-3300 


0  6  APR  1995 


MEMORANDUM  FOR  INSPECTOR  GENERAL,  DEPARTMENT  OF  DEFENSE 

SUBJECT:  Draft  Audit  Report  on  Joint  Cross-Service  Group  for 

Laboratories ,  1995  Base  Realignment  and  Closure 
Process  (Project  No.  4CG-5016.07) 


I  have  reviewed  the  draft  report  and  concur  in  the  auditor's 
description  of  the  process  used  by  the  Joint  Cross-Service  Group 
iJCSG)  for  Laboratories  to  develop  alternatives  for  consideration 
by  the  Military  Departments  during  their  BRAC  analyses. 

The  Inspector  General,  DoD,  has  been  a  key  part  of  the 
Department's  BRAC  process  by  providing  advice  and  review  of 
organizational  and  internal  management  controls  for  JCSG 
activities.  The  involvement  of  the  Inspector  General  enhanced 
the  process  by  helping  to  ensure  the  accuracy,  completeness,  and 
integrity  of  the  information  used  as  a  basis  for  development  of 
functional  alternatives  by  the  Joint  Cross-Service  Groups. 

Robert  E.  Bayer 

Deputy  Assistant  Secretary  of  Defense 
Installations 


Enclosure  3 


Audit  Team  Members 


Donald  E.  Reed 
Raymond  A.  Spencer 
David  F.  Vincent 
Nancee  K.  LaBute 
Richard  L.  Collier 
Kenneth  B.  Van  Hove 
Mary  Ann  Hourcle 
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INTERNET  DOCUMENT  INFORMATION  FORM 


A .  Report  Title:  Joint  Cross-Service  Group  for  Laboratories  1995  Defense 
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